Good or best practice statements: proposal for the operationalisation and implementation of GRADE guidance.

An evidence-based approach is considered the gold standard for health decision-making. Sometimes, a guideline panel might judge the certainty that the desirable effects of an intervention clearly outweigh its undesirable effects as high, but the body of supportive evidence is indirect. In such cases, the application of the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach for grading the strength of recommendations is inappropriate. Instead, the GRADE Working Group has recommended developing ungraded best or good practice statement (GPS) and developed guidance under which circumsances they would be appropriate.Through an evaluation of COVID-1- related recommendations on the eCOVID Recommendation Map (COVID-19.recmap.org), we found that recommendations qualifying a GPS were widespread. However, guideline developers failed to label them as GPS or transparently report justifications for their development. We identified ways to improve and facilitate the operationalisation and implementation of the GRADE guidance for GPS.Herein, we propose a structured process for the development of GPSs that includes applying a sequential order for the GRADE guidance for developing GPS. This operationalisation considers relevant evidence-to-decision criteria when assessing the net consequences of implementing the statement, and reporting information supporting judgments for each criterion. We also propose a standardised table to facilitate the identification of GPS and reporting of their development. This operationalised guidance, if endorsed by guideline developers, may palliate some of the shortcomings identified. Our proposal may also inform future updates of the GRADE guidance for GPS.

Methodological guidance for incorporating equity when informing rapid-policy and guideline development.

OBJECTIVES: We provide guidance for considering equity in rapid reviews through examples of published COVID-19 rapid reviews. STUDY DESIGN AND SETTING: This guidance was developed based on a series of methodological meetings, review of internationally renowned guidance such as the Cochrane Handbook and the Preferred Reporting Items for Systematic Reviews and Meta-Analysis for equity-focused systematic reviews (PRISMA-Equity) guideline. We identified Exemplar rapid reviews by searching COVID-19 databases and requesting examples from our team. RESULTS: We proposed the following key steps: 1. involve relevant stakeholders with lived experience in the conduct and design of the review; 2. reflect on equity, inclusion and privilege in team values and composition; 3. develop research question to assess health inequities; 4. conduct searches in relevant disciplinary databases; 5. collect data and critically appraise recruitment, retention and attrition for populations experiencing inequities; 6. analyse evidence on equity; 7. evaluate the applicability of findings to populations experiencing inequities; and 8. adhere to reporting guidelines for communicating review findings. We illustrated these methods through rapid review examples. CONCLUSION: Implementing this guidance could contribute to improving equity considerations in rapid reviews produced in public health emergencies, and help policymakers better understand the distributional impact of diseases on the population.

PROTOCOL: Barriers and facilitators to stakeholder engagement in health guideline development: A qualitative evidence synthesis.

Background: There is a need for the development of comprehensive, global, evidence-based guidance for stakeholder engagement in guideline development. Stakeholders are any individual or group who is responsible for or affected by health- and healthcare-related decisions. This includes patients, the public, providers of health care and policymakers for example. As part of the guidance development process, Multi-Stakeholder Engagement (MuSE) Consortium set out to conduct four concurrent systematic reviews to summarise the evidence on: (1) existing guidance for stakeholder engagement in guideline development, (2) barriers and facilitators to stakeholder engagement in guideline development, (3) managing conflicts of interest in stakeholder engagement in guideline development and (4) measuring the impact of stakeholder engagement in guideline development. This protocol addresses the second systematic review in the series. Objectives: The objective of this review is to identify and synthesise the existing evidence on barriers and facilitators to stakeholder engagement in health guideline development. We will address this objective through two research questions: (1) What are the barriers to multi-stakeholder engagement in health guideline development across any of the 18 steps of the GIN-McMaster checklist? (2) What are the facilitators to multi-stakeholder engagement in health guideline development across any of the 18 steps of the GIN-McMaster checklist? Search Methods: A comprehensive search strategy will be developed and peer-reviewed in consultation with a medical librarian. We will search the following databases: MEDLINE, Cumulative Index to Nursing & Allied Health Literature (CINAHL), EMBASE, PsycInfo, Scopus, and Sociological Abstracts. To identify grey literature, we will search the websites of agencies who actively engage stakeholder groups such as the AHRQ, Canadian Institutes of Health Research (CIHR) Strategy for Patient-Oriented Research (SPOR), INVOLVE, the National Institute for Health and Care Excellence (NICE) and the PCORI. We will also search the websites of guideline-producing agencies, such as the American Academy of Pediatrics, Australia's National Health Medical Research Council (NHMRC) and the WHO. We will invite members of the team to suggest grey literature sources and we plan to broaden the search by soliciting suggestions via social media, such as Twitter. Selection Criteria: We will include empirical qualitative and mixed-method primary research studies which qualitatively report on the barriers or facilitators to stakeholder engagement in health guideline development. The population of interest is stakeholders in health guideline development. Building on previous work, we have identified 13 types of stakeholders whose input can enhance the relevance and uptake of guidelines: Patients, caregivers and patient advocates; Public; Providers of health care; Payers of health services; Payers of research; Policy makers; Program managers; Product makers; Purchasers; Principal investigators and their research teams; and Peer-review editors/publishers. Eligible studies must describe stakeholder engagement at any of the following steps of the GIN-McMaster Checklist for Guideline Development. Data Collection and Analysis: All identified citations from electronic databases will be imported into Covidence software for screening and selection. Documents identified through our grey literature search will be managed and screened using an Excel spreadsheet. A two-part study selection process will be used for all identified citations: (1) a title and abstract review and (2) full-text review. At each stage, teams of two review authors will independently assess all potential studies in duplicate using a priori inclusion and exclusion criteria. Data will be extracted by two review authors independently and in duplicate according to a standardised data extraction form. Main Results: The results of this review will be used to inform the development of guidance for multi-stakeholder engagement in guideline development and implementation. This guidance will be official GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group guidance. The GRADE system is internationally recognised as a standard for guideline development. The findings of this review will assist organisations who develop healthcare, public health and health policy guidelines, such as the World Health Organization, to involve multiple stakeholders in the guideline development process to ensure the development of relevant, high quality and transparent guidelines.

An evaluation of the COVID-19 recommendation map identified diverging clinical and public health guidance.

OBJECTIVES: To describe divergence between actionable statements issued by coronavirus disease 2019 (COVID-19) guideline developers cataloged on the "COVID-19 Recommendations and Gateway to Contextualization" platform. STUDY DESIGN AND SETTING: We defined divergence as at least two comparable actionable statements with different explicit judgments of strength, direction, or subgroup consideration of the population or intervention. We applied a content analysis to compare guideline development methods for a sample of diverging statements and to evaluate factors associated with divergence. RESULTS: Of the 138 guidelines evaluated, 85 (62%) contained at least one statement that diverged from another guideline. We identified 223 diverging statements in these 85 guidelines. We grouped statements into 66 clusters. Each cluster addressed the same population, intervention, and comparator group or just similar interventions. Clinical practice statements were more likely to diverge in an explicit judgment of strength or direction compared to public health statements. Statements were more likely to diverge in strength than direction. The date of publication, used evidence, interpretation of evidence, and contextualization considerations were associated with divergence. CONCLUSION: More than half of the assessed guidelines issued at least one diverging statement. This study helps in understanding the types of differences between guidelines issuing comparable statements and factors associated with their divergence.

A multistakeholder development process to prioritize and translate COVID-19 health recommendations for patients, caregivers and the public. A case study of the COVID-19 recommendation map.

BACKGROUND AND OBJECTIVES: To develop a digital communication tool to improve the implementation of up-to-date COVID-19 recommendations. Specifically, to improve patient, caregiver and public understanding of healthcare recommendations on prevention, diagnoses and treatment. METHODS: Multi-stakeholder engagement design. In conjunction with the COVID-19 Recommendations and Gateway to Contextualization RecMap, we co-developed a stakeholder prioritization, drafting and editing process to enhance guideline communication and understanding. RESULTS: This paper presents the multi-stakeholder development process with three distinct plain language recommendation formats: formal recommendation, good practice statement, and additional guidance. Our case study of COVID-19 plain language recommendations PLRs addresses both public health interventions (e.g., vaccination, face masks) and clinical interventions (e.g., home pulse oximetry). CONCLUSION: This paper presents a novel approach to engaging stakeholders in improving the communication and understanding of published guidelines during the COVID-19 pandemic.

Which actionable statements qualify as good practice statements In Covid-19 guidelines? A systematic appraisal.

OBJECTIVES: To evaluate the development and quality of actionable statements that qualify as good practice statements (GPS) reported in COVID-19 guidelines. DESIGN AND SETTING: Systematic review . We searched MEDLINE, MedSci, China National Knowledge Infrastructure (CNKI), databases of Grading of Recommendations Assessment, Development and Evaluation (GRADE) Guidelines, NICE, WHO and Guidelines International Network (GIN) from March 2020 to September 2021. We included original or adapted recommendations addressing any COVID-19 topic. MAIN OUTCOME MEASURES: We used GRADE Working Group criteria for assessing the appropriateness of issuing a GPS: (1) clear and actionable; (2) rationale necessitating the message for healthcare practice; (3) practicality of systematically searching for evidence; (4) likely net positive consequences from implementing the GPS and (5) clear link to the indirect evidence. We assessed guideline quality using the Appraisal of Guidelines for Research and Evaluation II tool. RESULTS: 253 guidelines from 44 professional societies issued 3726 actionable statements. We classified 2375 (64%) as GPS; of which 27 (1%) were labelled as GPS by guideline developers. 5 (19%) were labelled as GPS by their authors but did not meet GPS criteria. Of the 2375 GPS, 85% were clear and actionable; 59% provided a rationale necessitating the message for healthcare practice, 24% reported the net positive consequences from implementing the GPS. Systematic collection of evidence was deemed impractical for 13% of the GPS, and 39% explained the chain of indirect evidence supporting GPS development. 173/2375 (7.3%) statements explicitly satisfied all five criteria. The guidelines' overall quality was poor regardless of the appropriateness of GPS development and labelling. CONCLUSIONS: Statements that qualify as GPS are common in COVID-19 guidelines but are characterised by unclear designation and development processes, and methodological weaknesses.

Mobile interventions targeting common mental disorders among pregnant and postpartum women: An equity-focused systematic review.

INTRODUCTION: Pregnant and postpartum women face major psychological stressors that put them at higher risk of developing common mental disorders, such as depression and anxiety. Yet, their limited access to and uptake of traditional mental health care is inequitable, especially during the COVID-19 pandemic. Mobile interventions emerged as a potential solution to this discontinued healthcare access, but more knowledge is needed about their effectiveness and impact on health equity. This equity-focused systematic review examined the effectiveness and equity impact of mobile interventions targeting common mental disorders among pregnant and postpartum women. METHODS AND RESULTS: We systematically searched MEDLINE, EMBASE, PsychINFO and 3 other databases, from date of database inception and until January 2021, for experimental studies on mobile interventions targeting pregnant and postpartum women. We used pooled and narrative synthesis methods to analyze effectiveness and equity data, critically appraised the methodological rigour of included studies using Cochrane tools, and assessed the certainty of evidence using the GRADE approach. Our search identified 6148 records, of which 18 randomized and non-randomized controlled trials were included. Mobile interventions had a clinically important impact on reducing the occurrence of depression (OR = 0.51 [95% CI 0.41 to 0.64]; absolute risk reduction RD: 7.14% [95% CI 4.92 to 9.36]; p<0.001) and preventing its severity perinatally (MD = -3.07; 95% CI -4.68 to -1.46; p<0.001). Mobile cognitive behavioural therapy (CBT) was effective in managing postpartum depression (MD = -6.87; 95% CI -7.92 to -5.82; p<0.001), whereas other support-based interventions had no added benefit. Results on anxiety outcomes and utilization of care were limited. Our equity-focused analyses showed that ethnicity, age, education, and being primiparous were characteristics of influence to the effectiveness of mobile interventions. CONCLUSION: As the COVID-19 pandemic has increased the need for virtual mental health care, mobile interventions show promise in preventing and managing common mental disorders among pregnant and postpartum women. Such interventions carry the potential to address health inequity but more rigorous research that examines patients' intersecting social identities is needed.

A taxonomy and framework for identifying and developing actionable statements in guidelines suggests avoiding informal recommendations.

OBJECTIVE: To propose a taxonomy and framework that identifies and presents actionable statements in guidelines. STUDY DESIGN AND SETTING: We took an iterative approach reviewing case studies of guidelines produced by the World Health Organization and the American Society of Hematology to develop an initial conceptual framework. We then tested it using randomly selected recommendations from published guidelines addressing COVID-19 from different organizations, evaluated its results, and refined it before retesting. The urgency and availability of evidence for development of these recommendations varied. We consulted with experts in research methodology and guideline developers to improve the final framework. RESULTS: The resulting taxonomy and framework distinguishes five types of actional statements: formal recommendations; research recommendations; good practice statements; implementation considerations, tools and tips; and informal recommendations. These statements should respond to a priori established criteria and require a clear structure and recognizable presentation in a guideline. Most importantly, this framework identifies informal recommendations that differ from formal recommendations by how they consider evidence and in their development process. CONCLUSION: The identification, standardization and explicit labelling of actionable statements according to the framework may support guideline developers to create actionable statements with clear intent, avoid informal recommendations and improve their understanding and implementation by users.

Improving Social Justice in COVID-19 Health Research: Interim Guidelines for Reporting Health Equity in Observational Studies.

The COVID-19 pandemic has highlighted the global imperative to address health inequities. Observational studies are a valuable source of evidence for real-world effects and impacts of implementing COVID-19 policies on the redistribution of inequities. We assembled a diverse global multi-disciplinary team to develop interim guidance for improving transparency in reporting health equity in COVID-19 observational studies. We identified 14 areas in the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist that need additional detail to encourage transparent reporting of health equity. We searched for examples of COVID-19 observational studies that analysed and reported health equity analysis across one or more social determinants of health. We engaged with Indigenous stakeholders and others groups experiencing health inequities to co-produce this guidance and to bring an intersectional lens. Taking health equity and social determinants of health into account contributes to the clinical and epidemiological understanding of the disease, identifying specific needs and supporting decision-making processes. Stakeholders are encouraged to consider using this guidance on observational research to help provide evidence to close the inequitable gaps in health outcomes.

Canadian Rheumatology Association Recommendation for the Use of COVID-19 Vaccination for Patients With Autoimmune Rheumatic Diseases.

OBJECTIVE: To develop guidance on the use of coronavirus disease 2019 (COVID-19) vaccines in patients with autoimmune rheumatic diseases (ARD). METHODS: The Canadian Rheumatology Association (CRA) formed a multidisciplinary panel including rheumatologists, researchers, methodologists, vaccine experts, and patients. The panel used the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach. Outcomes were prioritized according to their importance for patients and clinicians. Evidence from the COVID-19 clinical trials was summarized. Indirect evidence for non-COVID-19 vaccines in ARD was also considered. The GRADE evidence-to-decision (EtD) framework was used to develop a recommendation for the use of the 4 COVID-19 vaccines approved in Canada as of March 25, 2021 (BNT162b2, mRNA-1273, ChAdOx1, and Ad26.COV2.S), over 4 virtual panel meetings. RESULTS: The CRA guideline panel suggests using COVID-19 vaccination in persons with ARD. The panel unanimously agreed that for the majority of patients, the potential health benefits of vaccination outweigh the potential harms in people with ARDs. The recommendation was graded as conditional because of low or very low certainty of the evidence on the effects in the population of interest, primarily due to indirectness and imprecise effect estimates. The panel felt strongly that persons with autoimmune rheumatic diseases who meet local eligibility should not be required to take additional steps compared to people without ARDs to obtain their vaccination. Guidance on medications, implementation, monitoring of vaccine uptake, and research priorities are also provided. CONCLUSION: This recommendation will be updated over time as new evidence emerges, with the latest recommendation, evidence summaries, and EtD available on the CRA website.